Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(mempool): refactor test_tip_priority_over_tx_hash to use mempool… #446

Conversation

ayeletstarkware
Copy link
Contributor

@ayeletstarkware ayeletstarkware commented Jul 11, 2024

… state


This change is Reviewable

@codecov-commenter
Copy link

codecov-commenter commented Jul 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.21%. Comparing base (f1d94c0) to head (757a6c0).

Additional details and impacted files
@@                          Coverage Diff                           @@
##           ayelet/mempool/mempool-state/refactor     #446   +/-   ##
======================================================================
  Coverage                                  81.21%   81.21%           
======================================================================
  Files                                         42       42           
  Lines                                       1826     1826           
  Branches                                    1826     1826           
======================================================================
  Hits                                        1483     1483           
  Misses                                       269      269           
  Partials                                      74       74           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@MohammadNassar1 MohammadNassar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @ayeletstarkware, @elintul, and @giladchase)


crates/mempool/src/mempool_test.rs line 249 at r1 (raw file):

    let mempool_state = MempoolState::new(pool_txs, queue_txs);

    mempool_state.assert_eq_mempool_state(&mempool);

Suggestion:

    let pool_txs = [input_big_tip_small_hash.tx, input_small_tip_big_hash.tx];
    
    let mempool_state = MempoolState::new(pool_txs, queue_txs);
    mempool_state.assert_eq_mempool_state(&mempool);

@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool-state-compare-func branch from ba6056d to 7479fd6 Compare July 14, 2024 08:23
@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool_state/test-tip-priority-over-tx-hash branch from 85434e3 to 3d75e4f Compare July 14, 2024 09:34
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @elintul, @giladchase, and @MohammadNassar1)


crates/mempool/src/mempool_test.rs line 249 at r1 (raw file):

    let mempool_state = MempoolState::new(pool_txs, queue_txs);

    mempool_state.assert_eq_mempool_state(&mempool);

I think the first option is more readable

Copy link
Contributor

@MohammadNassar1 MohammadNassar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 2 unresolved discussions (waiting on @ayeletstarkware, @elintul, and @giladchase)


crates/mempool/src/mempool_test.rs line 256 at r2 (raw file):

#[rstest]
fn test_tip_priority_over_tx_hash(mut mempool: Mempool) {

What we are testing here?
I don't see get_txs method.

Code quote:

n test_tip_priority_over_tx_hash(

Copy link
Contributor

@MohammadNassar1 MohammadNassar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @ayeletstarkware, @elintul, and @giladchase)


crates/mempool/src/mempool_test.rs line 256 at r2 (raw file):

Previously, MohammadNassar1 (mohammad-starkware) wrote…

What we are testing here?
I don't see get_txs method.

Please ignore.
The test tests that in tx_queue there are two transactions, and not one, as they have different tx hash.

@ayeletstarkware ayeletstarkware changed the base branch from ayelet/mempool/mempool-state-compare-func to main July 15, 2024 06:46
@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool_state/test-tip-priority-over-tx-hash branch 2 times, most recently from adb7e84 to 89971b8 Compare July 15, 2024 07:16
Copy link
Collaborator

@giladchase giladchase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @ayeletstarkware, @elintul, and @MohammadNassar1)


crates/mempool/src/mempool_test.rs line 273 at r3 (raw file):

        TransactionReference::new(&input_small_tip_big_hash.tx),
    ];
    let expected_pool_txs = [input_big_tip_small_hash.tx, input_small_tip_big_hash.tx];

Unnecessary in this test, there are only 2 txs and both are in the queue.

@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool_state/test-tip-priority-over-tx-hash branch from 89971b8 to 47f70b3 Compare July 15, 2024 15:33
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @elintul, @giladchase, and @MohammadNassar1)


crates/mempool/src/mempool_test.rs line 273 at r3 (raw file):

Previously, giladchase wrote…

Unnecessary in this test, there are only 2 txs and both are in the queue.

Done.

Copy link
Collaborator

@giladchase giladchase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @ayeletstarkware, @elintul, and @MohammadNassar1)


crates/mempool/src/mempool_test.rs line 44 at r4 (raw file):

        let tx_queue: TransactionQueue = queue_txs.into_iter().collect();
        MempoolState { tx_pool: Default::default(), tx_queue }
    }

This is a good attempt for what we have now, and i nearly LGTMed it for now, but this might be hard to debug when a test fails: one would typically print the mempool contents when debugging, which will have an empty pool, and that can be very confusing to anyone not intimately aware of this util.

So I think a more strongly typed approach is in order, even at this stage.
See if this works:

#[derive(Debug)]
struct MempoolState<T> {
    tx_pool: Option<TransactionPool>,
    tx_queue: Option<TransactionQueue>,
    _phantom: PhantomData<T>,
}

#[derive(Debug)]
struct FullState;
#[derive(Debug)]
struct PartialState;

impl MempoolState<FullState> {
    fn new<P, Q>(pool_txs: P, queue_txs: Q) -> Self
    where
        P: IntoIterator<Item = ThinTransaction>,
        Q: IntoIterator<Item = TransactionReference>,
    {
        Self {
            tx_pool: Some(pool_txs.into_iter().collect()),
            tx_queue: Some(queue_txs.into_iter().collect()),
            _phantom: PhantomData,
        }
    }
}

impl Default for MempoolState<PartialState> {
    fn default() -> Self {
        Self { tx_pool: None, tx_queue: None, _phantom: PhantomData }
    }
}

impl MempoolState<PartialState> {
    fn empty() -> Self {
        Default::default()
    }

    fn _with_pool<P>(mut self, pool_txs: P) -> Self
    where
        P: IntoIterator<Item = ThinTransaction>,
    {
        self.tx_pool = Some(pool_txs.into_iter().collect());
        self
    }

    fn with_queue<Q>(mut self, queue_txs: Q) -> Self
    where
        Q: IntoIterator<Item = TransactionReference>,
    {
        self.tx_queue = Some(queue_txs.into_iter().collect());
        self
    }
}

impl<T> MempoolState<T> {
    fn assert_eq_mempool_state(&self, mempool: &Mempool) {
        self.assert_eq_pool_state(mempool);
        self.assert_eq_queue_state(mempool);
    }

    fn assert_eq_pool_state(&self, mempool: &Mempool) {
        assert_eq!(self.tx_pool.as_ref().unwrap(), &mempool.tx_pool);
    }

    fn assert_eq_queue_state(&self, mempool: &Mempool) {
        assert_eq!(self.tx_queue.as_ref().unwrap(), &mempool.tx_queue);
    }
}

If it works, and it is clear how to use, use it instead, otherwise lets try to think of another solution that will print None when debugging these kinds of tests, but that doesn't incur boilerplate during the tests, and that can also scale to the remaining components we'll soon need here (like the so-called "staging area")

Code quote:

    fn new_with_queue_txs_default_queue<QueueTxs>(queue_txs: QueueTxs) -> Self
    where
        QueueTxs: IntoIterator<Item = TransactionReference>,
    {
        let tx_queue: TransactionQueue = queue_txs.into_iter().collect();
        MempoolState { tx_pool: Default::default(), tx_queue }
    }

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @giladchase and @MohammadNassar1)


crates/mempool/src/mempool_test.rs line 44 at r4 (raw file):

Previously, giladchase wrote…

This is a good attempt for what we have now, and i nearly LGTMed it for now, but this might be hard to debug when a test fails: one would typically print the mempool contents when debugging, which will have an empty pool, and that can be very confusing to anyone not intimately aware of this util.

So I think a more strongly typed approach is in order, even at this stage.
See if this works:

#[derive(Debug)]
struct MempoolState<T> {
    tx_pool: Option<TransactionPool>,
    tx_queue: Option<TransactionQueue>,
    _phantom: PhantomData<T>,
}

#[derive(Debug)]
struct FullState;
#[derive(Debug)]
struct PartialState;

impl MempoolState<FullState> {
    fn new<P, Q>(pool_txs: P, queue_txs: Q) -> Self
    where
        P: IntoIterator<Item = ThinTransaction>,
        Q: IntoIterator<Item = TransactionReference>,
    {
        Self {
            tx_pool: Some(pool_txs.into_iter().collect()),
            tx_queue: Some(queue_txs.into_iter().collect()),
            _phantom: PhantomData,
        }
    }
}

impl Default for MempoolState<PartialState> {
    fn default() -> Self {
        Self { tx_pool: None, tx_queue: None, _phantom: PhantomData }
    }
}

impl MempoolState<PartialState> {
    fn empty() -> Self {
        Default::default()
    }

    fn _with_pool<P>(mut self, pool_txs: P) -> Self
    where
        P: IntoIterator<Item = ThinTransaction>,
    {
        self.tx_pool = Some(pool_txs.into_iter().collect());
        self
    }

    fn with_queue<Q>(mut self, queue_txs: Q) -> Self
    where
        Q: IntoIterator<Item = TransactionReference>,
    {
        self.tx_queue = Some(queue_txs.into_iter().collect());
        self
    }
}

impl<T> MempoolState<T> {
    fn assert_eq_mempool_state(&self, mempool: &Mempool) {
        self.assert_eq_pool_state(mempool);
        self.assert_eq_queue_state(mempool);
    }

    fn assert_eq_pool_state(&self, mempool: &Mempool) {
        assert_eq!(self.tx_pool.as_ref().unwrap(), &mempool.tx_pool);
    }

    fn assert_eq_queue_state(&self, mempool: &Mempool) {
        assert_eq!(self.tx_queue.as_ref().unwrap(), &mempool.tx_queue);
    }
}

If it works, and it is clear how to use, use it instead, otherwise lets try to think of another solution that will print None when debugging these kinds of tests, but that doesn't incur boilerplate during the tests, and that can also scale to the remaining components we'll soon need here (like the so-called "staging area")

Nice! What's the use of PhantomData?

Copy link
Collaborator

@giladchase giladchase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @elintul and @MohammadNassar1)


crates/mempool/src/mempool_test.rs line 44 at r4 (raw file):

Previously, elintul (Elin) wrote…

Nice! What's the use of PhantomData?

Because we don't really use FullState and PartialState, they are just "markers" which are just some type artifact that allows the compiler to treat MempoolState<FullState> and MempoolState<PartialState> as separate types.

Without the phantomdata field, the compiler would have whined about there being an unused generic T in the definition of Mempool.
It is also optimized for this specific purpose, and in particular is called a "zero sized type", because it has 0 size on the stack. In other words, when a mempool instance is initialized and put on the stack, only the pool/queue fields take up space, whereas the phantomdata field doesn't appear.

However, if you dbg!(mempool_state), you will see MempoolState or MempoolState (depending on which one you printed), because the compiler treats these as separate types

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @giladchase and @MohammadNassar1)


crates/mempool/src/mempool_test.rs line 44 at r4 (raw file):

Previously, giladchase wrote…

Because we don't really use FullState and PartialState, they are just "markers" which are just some type artifact that allows the compiler to treat MempoolState<FullState> and MempoolState<PartialState> as separate types.

Without the phantomdata field, the compiler would have whined about there being an unused generic T in the definition of Mempool.
It is also optimized for this specific purpose, and in particular is called a "zero sized type", because it has 0 size on the stack. In other words, when a mempool instance is initialized and put on the stack, only the pool/queue fields take up space, whereas the phantomdata field doesn't appear.

However, if you dbg!(mempool_state), you will see MempoolState or MempoolState (depending on which one you printed), because the compiler treats these as separate types

Very cool, thanks. Let's try to imp. this in a separate PR.

@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool_state/test-tip-priority-over-tx-hash branch 2 times, most recently from 00b48c5 to 4fe1235 Compare July 18, 2024 12:20
@ayeletstarkware ayeletstarkware changed the base branch from main to ayelet/mempool/mempool-state/refactor July 18, 2024 12:21
Copy link
Contributor

@MohammadNassar1 MohammadNassar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r5, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ayeletstarkware)

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r5, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @ayeletstarkware)


crates/mempool/src/mempool_test.rs line 351 at r6 (raw file):

    add_tx(&mut mempool, &input_small_tip_big_hash);

    // Assert: ensure that the transaction with the higher tip is sequenced first.

Suggestion:

prioritized higher

@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool-state/refactor branch from 6b70786 to f5a081b Compare July 24, 2024 08:17
@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool_state/test-tip-priority-over-tx-hash branch from 4fe1235 to d91ace9 Compare July 24, 2024 08:20
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r7, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ayeletstarkware)

@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool-state/refactor branch 2 times, most recently from 4050a0a to f1d94c0 Compare July 25, 2024 07:00
@ayeletstarkware ayeletstarkware force-pushed the ayelet/mempool/mempool_state/test-tip-priority-over-tx-hash branch from d91ace9 to 757a6c0 Compare July 25, 2024 09:10
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r8, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ayeletstarkware)

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ayeletstarkware)

@ayeletstarkware ayeletstarkware merged commit 232db3d into ayelet/mempool/mempool-state/refactor Jul 25, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants